Learning Task Specific Distributed Paragraph Representations Using a 2-Tier Convolutional Neural Network

We introduce a type of two-tier convolutional neural network model for learning distributed paragraph representations for a special task (e.g. topic classification or sentiment classification). We decompose the paragraph semantics into three cascaded constitutes: word representation, sentence composition and document composition. Specifically, we learn distributed word representations by a continuous bag-of-words model from a large unstructured text corpus. Then, using these word representations as pre-trained vectors, the first tier of our model learns the distributed task-specific sentence representations from a corpus where each sentence is annotated with a task-specific label. Subsequently, the second tier of our model learns the distributed paragraph representations of a different document corpus from the learned sentence representations. Our proposed model has been evaluated on topic classification based on the DBpedia ontology and sentiment classification of Amazon reviews. Empirical results show the effectiveness of our proposed learning model for generating distributed paragraph representations.